The problem of a Digital Dark Ages looms before historians, archivists, librarians, and governments. Today, the functioning of our society relies almost completely upon vast numbers of documents that are stored digitally, yet digital records barely five or ten years old may be extremely difficult to access if they are stored on media that in the future falls out of favor. The situation over the long term is bleaker still, and there is a very real possibility that the vast majority of the records of our civilization, in particular, records stored in digital form, may not last more than 100 years. Digital media will not endure (bit rot)—magnetic storage fades, CDs fade and decompose, drive mechanisms corrode, and non-volatile RAM fade. Additionally, old formats will be lost (format rot)—file formats will be forgotten, encryption and compression schemes will be lost to memory, and PROMs will be indecipherable once the documentation or hardware systems are lost. Finally, equipment and components that are necessary to read old media may become obsolete and unavailable in the commercial market if they fail to maintain market share, and it is not possible to predict which technologies may fall out of favor at any far-off future date.
A prime example of the problem is the BBC Domesday Book, a re-publication of the original Domesday Book census of 11th-century England. The BBC update added to this record a great deal of information on the modern populace of Britain, with text, audio, video, and interactive programs. It was published in 1986 as a set of two laserdiscs, totaling one gigabyte (GB). Although the original record has endured in a readable format for over 900 years, the modern version became virtually impossible to access in only 15 years. This was due to the advance of digital standards and the failure of old and obsolete equipment.
Library and governmental institutions have an imperative to implement a media refreshing/migration program to counteract obsolescence and many institutions have sustainable funding provided specifically for long-term maintenance of digital data. A. R. Kenney, “Developing Digital Preservation Programs: the Cornell Survey of Institutional Readiness, 2003-2005,” Res. Libraries Group DigiNews 9, n4, Art1 (2005). However, the media refreshing/migration program approach invites recurring costs and losses due to repeated migrations. A single solution for once-only archiving is desperately needed.
Digital storage is also susceptible to manipulation (e.g., alteration, corruption) when stored in rewritable formats, as all high-volume backup media such as magnetic tape are. Technologies capable of implementing legislated digital-information archiving requirements (such as Check21, 21 CFR Part 11, DoD5015.2, and HIPAA) do not exist. In fact, no permanent digital archival technology exists.
All existing digital storage media rely upon a specialized mechanical device for readout. Availability of a functioning readout device is necessary to retrieve any stored information.
CD-Rs and DVD-Rs are a commonly used for backup of digital data by 70% of library and government agencies. Id. These articles of media are based on organic dyes (pthalo-cyanine, cyanine, and azo) with a silver alloy reflective layer or layers in a polycarbonate body. These organic dyes fade naturally over time (O. Slattery, R. Lu, J. Zheng, F. Byers, and X. Tang, “Stability Coparison of Recordable Optical Discs—A Study of Error Rates in Harsh Conditions,” J. Res. Natl. Inst. Stand. Technol. 109, 517 (2004)), and this kinetic process is accelerated by high humidity and temperature in aging tests, rapidly resulting in faded dyes, delaminations, and unreadable data (J. Iraci, “The Relative Stabilities of Optical Disc Formats,” Restaurator: International Journal for the Preservation of Library and Archival Material 26, 134 (2005)). Degradation is often due to oxygen and water vapor entering through imperfections in the edge, through the polycarbonate body, or the lacquer coating, with the resulting oxidation and destruction of the data storage layer or the reflective layer. F. R. Byers, “Care and Handling of CDs and DVDs: A Guide for Librarians and Archivists,” NIST (2003). The dyes are light-sensitive, and storage in a lit room accelerates aging. Manufacturers claim a shelf life of only 5 to 10 years for unrecorded discs. CD-Rs have a typical lifetime of 2 to 10 years, although manufacturers sometimes make unsupported claims of 100 years. Tests indicate that pthalocyanine CD-Rs are superior to other optical discs, including pthalocyanine DVD-Rs, which exhibit shorter readable-data lifetimes. See, Iraci. The internal error correction software of CD/DVD readers can mask read problems as a disc decays and bit error rates increase, with the user becoming aware only when a read failure occurs. Storage in a dark, dry nitrogen atmosphere can extend the lifetime, but incurs continual extra archiving expense.
Hard disks have high data storage reliability, but are integrated with a mechanical system. Over decades, the mechanical system is likely to degrade, and failure of that system, such as dry-out of the spindle bearing, results in loss of all data on the device.
Magneto-optical (MO) discs were touted as an archival medium several years ago, but the failure of that technology to be adopted widely has led to the obsoleting of the technology for most potential users. Today, only 20 years after their introduction, a working MO player is difficult to find in most areas, the media have become extraordinarily expensive, and use is limited to medical applications.
Magnetic tapes, stored in metal canisters, have magnetization lifetimes predicted to be 100 years if stored at 30% RH and ambient temperatures. R. D. Weiss et al., “Environmental Stability Study and Life Expectancies of Magnetic Media for Use with IBM 3590 and Quantum Digital Linear Tape Systems,” Final Report to the National Archives and Records Administration, NAMA-01-F-0061 (2002). However, the binders for the magnetic particles can degrade more quickly. Id.
Flash drives are quite common today, but rely upon a trapped charge as a mechanism of storage, and are not expected to have any lifetime adequate for archiving or even backup.
Internet-based backup services are currently quite popular, but such systems are susceptible to any interruption. And in any case, these systems in the end must store the digital data in some form, such as hard drives or tape.
It would be helpful to be able to provide a digital data preservation technology that addresses one or more of the deficiencies of prior data storage technologies. Also, it would be helpful to be able to provide a digital data preservation technology that does not rely upon any specialized hardware.